Skip to content

feat: test against wit-bingen tests#16

Merged
GordonSmith merged 1 commit intomainfrom
MORE_TESTS
Oct 13, 2025
Merged

feat: test against wit-bingen tests#16
GordonSmith merged 1 commit intomainfrom
MORE_TESTS

Conversation

@GordonSmith
Copy link
Owner

@GordonSmith GordonSmith commented Oct 13, 2025

No description provided.

@GordonSmith GordonSmith requested a review from Copilot October 13, 2025 07:04
Copilot AI reviewed Oct 13, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR implements comprehensive testing and validation for the wit-codegen tool by integrating test files from the wit-bindgen test suite. The PR adds substantial infrastructure for testing code generation against 199 real-world WIT files, providing crucial quality assurance for the code generator.

  • WIT-bindgen test integration: Comprehensive testing against 199 WIT files from the official wit-bindgen test suite
  • Modular architecture: Refactored wit-codegen.cpp into smaller, maintainable modules with improved type handling
  • Complete validation framework: Python scripts and CMake targets for generating and validating C++ stubs from WIT files

Reviewed Changes

Copilot reviewed 29 out of 31 changed files in this pull request and generated 5 comments.

Show a summary per file
File Description
validation_results.txt Test results showing 54.3% success rate across 199 WIT files with detailed failure analysis
tools/wit-codegen/wit-codegen.cpp Major refactoring with modular headers, enhanced type mapping, and improved WIT parsing
tools/wit-codegen/utils.hpp Utility functions for identifier sanitization and WIT parsing helpers
tools/wit-codegen/types.hpp Data structures for WIT AST representation and code generation
tools/wit-codegen/type_mapper.hpp Interface for WIT to C++ type mapping system
tools/wit-codegen/type_mapper.cpp Implementation of comprehensive type mapping with cross-namespace support
tools/wit-codegen/CMakeLists.txt Updated build configuration for modular architecture
test/validate_stubs.py Python script for incremental validation of generated code
test/validate_all_wit_bindgen.py Comprehensive validation against all wit-bindgen test files
test/generate_test_stubs.sh Bash script for batch stub generation
test/generate_test_stubs.py Enhanced Python script for stub generation with filtering
test/TESTING_GRAMMAR.md Updated documentation for grammar testing workflow
test/StubGenerationTests.cmake CMake integration for automated stub generation and validation
multiple documentation files Comprehensive guides for stub generation, validation, and incremental testing
include/cmcpp/*.hpp Enhanced type system with monostate support, record handling, and boolean operations

@GordonSmith GordonSmith requested a review from Copilot October 13, 2025 08:38
Copilot AI reviewed Oct 13, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 39 out of 41 changed files in this pull request and generated 3 comments.

Comments suppressed due to low confidence (1)

include/cmcpp/record.hpp:1

  • Header guard comment should match the opening guard. It references 'CMCPP_MONOSTATE_HPP' but the opening guard is 'CMCPP_RECORD_HPP'.
#ifndef CMCPP_RECORD_HPP

@codecov
Copy link

codecov bot commented Oct 13, 2025 •
edited
Loading

Codecov Report

❌ Patch coverage is 92.85714% with 2 lines in your changes missing coverage. Please review.
✅ Project coverage is 78.13%. Comparing base (5e1584f) to head (968dbae).
⚠️ Report is 1 commits behind head on main.

Files with missing lines Patch % Lines
include/cmcpp/monostate.hpp 80.00% 1 Missing ⚠️
include/cmcpp/variant.hpp 50.00% 1 Missing ⚠️
Additional details and impacted files 
@@            Coverage Diff             @@
##             main      #16      +/-   ##
==========================================
- Coverage   84.11%   78.13%   -5.98%     
==========================================
  Files          19       21       +2     
  Lines        1448     1537      +89     
  Branches      129      131       +2     
==========================================
- Hits         1218     1201      -17     
- Misses        140      236      +96     
- Partials       90      100      +10

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

@GordonSmith GordonSmith requested a review from Copilot October 13, 2025 09:11
Copilot AI reviewed Oct 13, 2025
View reviewed changes
Copy link
Contributor

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

Copilot reviewed 40 out of 42 changed files in this pull request and generated 1 comment.

@GordonSmith GordonSmith force-pushed the MORE_TESTS branch 2 times, most recently from db35a1a to 9a1f01d Compare October 13, 2025 09:51
@GordonSmith
feat: test against wit-bingen tests
968dbae 
Signed-off-by: Gordon Smith <GordonJSmith@gmail.com>
@github-actions
Copy link

github-actions bot commented Oct 13, 2025

Ubuntu Test Results

wit-stub-generation-test Results

This test is expected to have failures due to known issues with tuple wrapper types.

Compilation Summary

Metric Value
Total targets 37
✅ Passed 35 (94.6%)
❌ Failed 2 (5.4%)
Total errors 114

Error Breakdown

Error Type Count
std::get<> function call errors 0
result_ok_wrapper related 144
result_err_wrapper related 144
Missing file errors 2

Sample Errors

/home/runner/work/component-model-cpp/component-model-cpp/test/../include/cmcpp/variant.hpp:32:48: error: no match for ‘operator=’ (operand types are ‘std::variant<std::monostate, std::monostate>’ and ‘std::monostate’)
/home/runner/work/component-model-cpp/component-model-cpp/test/../include/cmcpp/variant.hpp:32:48: error: no match for ‘operator=’ (operand types are ‘std::variant<std::monostate, unsigned int, std::__cxx11::basic_string<char, std::char_traits<char>, std::allocator<char> >, cmcpp::record_t<host::variants::empty_data>, std::monostate, unsigned int, float>’ and ‘std::monostate’)
/usr/include/c++/13/tuple:2303:42: error: no matching function for call to ‘get<0>(cmcpp::result_err_wrapper<std::tuple<unsigned int> >&)’
/usr/include/c++/13/tuple:2303:42: error: no matching function for call to ‘get<0>(cmcpp::result_ok_wrapper<std::tuple<unsigned int> >&)’
/usr/include/c++/13/tuple:2303:42: error: type/value mismatch at argument 1 in template parameter list for ‘template<class _Tp, class ... _Types> constexpr _Tp& std::get(tuple<_UTypes ...>&)’

Known Issues

The main issue is that result_ok_wrapper and result_err_wrapper types don't support std::get<> operations needed for tuple unpacking in generated code.

Goal: These failures should ideally decrease over time or remain stable. Any increase warrants investigation.

Updated: 2025-10-13T10:05:46.183Z

@GordonSmith GordonSmith merged commit ce2cad4 into main Oct 13, 2025
3 of 4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

Copilot code review Copilot Copilot left review comments

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@GordonSmith